UNCLASSIFIED 


Defense  Technical  Information  Center 
Compilation  Part  Notice 

ADPO 10693 

TITLE:  The  Electronic  Sandtable:  An  Application 
of  VE-Technology  as  Tactical  Situation  Display 

DISTRIBUTION:  Approved  for  public  release,  distribution  unlimited 


This  paper  is  part  of  the  following  report: 

TITLE:  Usability  of  Information  in  Battle 
Management  Operations  [V Exploitation  de 
V information  dans  les  operations  de  gestion  du 
champ  de  bataille] 

To  order  the  complete  compilation  report,  use:  AD  A3 89629 

The  component  part  is  provided  here  to  allow  users  access  to  individually  authored  sections 
of  proceedings,  annals,  symposia,  ect.  However,  the  component  should  be  considered  within 
the  context  of  the  overall  compilation  report  and  not  as  a stand-alone  technical  report. 


The  following  component  part  numbers  comprise  the  compilation  report: 
ADPO  10683  thru  ADPO  10703 


UNCLASSIFIED 


12-1 


The  Electronic  Sandtable: 

An  Application  of  VE-Technology 
as  Tactical  Situation  Display. 


Thomas  Alexander 
Klaus-Peter  Gartner 

Research  Establishment  for  Applied  Science  (FGAN) 

Research  Institute  for  Communication,  Information  Processing  and  Ergonomics 
Dept.  Ergonomics  and  Information  Systems 
Neuenahrer  StraBe  20 
53343  Wachtberg-Werthhoven  / Germany 
Tel.:  +49  228  9435  -480 
Fax:  +49  228  9435  -508 
email:  alex@fgan.de,  gaertner@fgan.de 


SUMMARY 

In  future  battlespace  scenarios  huge  amounts  of  highly 
dynamic  information  will  be  available  due  to  the 
technical  development  of  sensor,  communication  and 
information  systems.  This  flood  of  available  information 
may  lead  to  mental  overload  of  the  military  commander 
and  cause  a wrong  mental  model  of  the  battlespace 
situation.  Therefore  advanced  techniques  for  supporting 
the  military  commander  and  displaying  complex  tactical 
situation  data  in  a clearly  understandable  way  have  to  be 
developed  and  evaluated. 

At  the  Research  Institute  for  Communication,  Infor- 
mation Processing  and  Ergonomics  (FKIE)  of  FGAN  a 
concept  for  preprocessing  and  visualizing  incoming 
tactical  data  and  three-dimensional  geographical  data  has 
been  developed.  The  concept  includes  the  use  of  Virtual 
Environment-technology  as  a display  system.  This 
“Electronic  Sandtable  (ELSA)”  testbed,  as  described  in 
this  paper,  is  based  on  a semi-immersive  display 
technology.  It  facilitates  a plastic  stereoscopic 
visualization  of  three-dimensional  data.  It  has  been 
developed  to  be  used  to  simulate  a sandtable  as 
commonly  used  by  the  Armed  Forces  for  tactical 
education  and  training. 

This  paper  presents  the  baseline  concept  of  using  VE- 
technology  as  an  advanced  tactical  situation  display.  It  is 
pointed  out  that,  although  the  technology  is 
commercially  available,  research  in  the  area  of 
ergonomics  and  human  factors  is  essential  for  the 
advantageous  use  of  such  a system.  The  main  ergonomic 
topics  described  in  this  paper  include  the  stereoscopic 
visualization  of  the  geographic  and  tactical  data,  the 
degree  of  abstraction  and  human  operator  interaction 
with  the  virtual  scene  on  the  “Electronic  Sandtable”. 

1 Introduction 

Within  the  scope  of  future  scenarios  there  will  be  a high 
demand  on  detailed  and  highly  actual  information  in 
military  command  and  control  (C1 2).  The  demand  will  be 
met  by  complex  information  databanks,  new  sensor 
technology  and  fast  electronic  communication  structures. 
Broad  data  acquisition,  transfer  and  presentation  will 


enable  the  military  commander  to  get  a variety  of  diverse 
information  about  the  battlefield  situation.  The 
accomplished  information  dominance  is  more  and  more 
considered  to  be  essential  for  a battlespace  dominance. 
However,  the  massive  quantity  of  information  is  also 
hazardous.  Especially  in  time-critical  situations  when 
tactical  decision  making  under  stress  is  required, 
relevant  information  may  be  overseen  and  a wrong 
mental  model  of  the  tactical  situation  is  gained.  That 
overload  is  likely  to  be  reduced  by  using  new 
technologies  for  data  preprocessing  and  data 
presentation.  Because  data  presentation  is  of  critical 
importance  in  the  whole  process  of  decision  making, 
ergonomic  research  is  required  to  analyze  the  whole 
process  of  data  presentation,  considering  new  displays 
and  interaction  devices. 

Especially  using  Virtual  Environment  (VE)-technology 
is  promising.  It  was  found  to  have  high  potential  in 
presenting  and  interacting  with  complex  amounts  of 
data.  Therefore  VE  will  increase  the  clearness  and 
intelligibility  of  a complex  tactical  situation.  The 
situation  scenario  is  not  perceived  as  a complex  of 
abstract  information  but  as  a pseudo-realistic  model 
landscape.  This  is  intensified  by  an  intuitive,  easy  to 
learn  interaction  with  the  included  objects. 

2 Definition  of  Virtual  Environment  (VE) 

The  basic  idea  of  generating  and  using  a computer- 
generated artificial  reality  was  mentioned  first  in  science 
fiction  literature  at  the  middle  of  the  20th  century.  Due  to 
rapid  development  of  computer  technology  in  the  second 
half  of  the  last  century,  a partly  realization  of  this  idea 
became  possible.  Nowadays  these  VE-Systems  are 
commercially  available  and  starting  to  be  used  for  a 
broad  range  of  applications  (Alexander  et  al,  1999). 
According  to  Bullinger  et  al  (1997),  Virtual  Environ- 
ments (VE)  describe  the  computer-based  generation  of 
an  intuitively  perceivable  and  experiencable  scene  of  a 
natural  or  an  abstract  environment.  It  is  characterized  by 
capacities  for  multi-modal,  three-dimensional  modeling 
and  simulation  of  objects  and  situations.  A further 
characteristic  is  the  close  interaction  of  the  human 
operator  with  the  system  . 


Paper  presented  al  the  RTO  HFM  Symposium  on  "Usability  of  Information  in  Battle  Management 
Operations’’,  held  in  Oslo,  Norway,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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In  this  connection,  Virtual  Reality  (VR),  has  been 
defined  by  NATO  HFM-021  (n.n.)  as 
"...  the  experience  of  being  in  a synthetic 
environment  and  the  perceiving  and  interacting 
through  sensors  and  effectors,  actively  and 
passively,  with  it  and  the  objects  in  it,  as  if  they  were 
real.  Virtual  Reality  technology  allows  the  user  to 
perceive  and  experience  sensory  contact  and 
interact  dynamically  with  such  contact  in  any  or  all 
modalities.  ” 

This  definition  of  VR  which  is  often  used  as  a synonym 
to  VE  overlaps  with  VE.  But  whereas  VE  is  application 
oriented,  VR  describes,  strictly  speaking,  a total  model 
of  the  reality,  including  all  manifold  facets  of  it.  As  this 
is  not  possible  today  and  may  not  be  possible  in  future, 
the  further  article  will  be  use  the  term  VE. 

VE  can  be  divided  into  at  least  three  groups  (Bullinger  ct 
al,  1997): 

• full-immersive  VE  is  characterized  by  a complete 
replacement  of  the  reality  by  the  virtual  reality.  The 
operator  is  completely  included  in  the  virtual  world 
and  does  not  perceive  any  (visual,  acoustic)  stimuli 
from  the  real  word.  A head-mounted  display  (HMD) 
is  a typical  full-immersive  display. 

• semi-immersive  VE:  The  virtual  scene  is  presented  as 
a three-dimensional  part  of  the  reality.  The  operator 
perceives  stimuli  from  the  real  world  and  additional 
stimuli  from  the  virtual  world.  He  cannot  distinguish 
between  real  and  virtual  objects.  Typical  semi- 
immersive  VE-Systems  are  workbenches  which  will 
be  described  later,  and  flight  simulators. 

• Desktop  VE:  The  three-dimensional  scene  is 
presented  on  a two-dimensional  display  medium.  Just 
interaction  and  navigation  happen  more  intuitively. 
VRML-Browsers  and  Videogames  work  like  this. 

VE-systems  are  on  their  way  of  becoming  used  for 
different  applications.  Further  information  about  military 
applications  is  given  in  Alexander  et  al  (1999). 

First,  VE  is  a research  topic  itself.  This  involves  basic 
research  studies  in  computer  science  as  well  as  interface 
design  and  ergonomics. 

Secondly  VE  is  used  as  a research  and  development 
tool.  In  this  area  VE  Systems  enable  a very  intensive  and 
direct  interaction  with  complex  and  abstract  data.  They 
enable  new  kinds  of  rapid  prototyping.  In  an  early  stage 
of  the  design  process  CAD-models  of  products  can  be 
visualized  and  examined  as  if  they  were  real. 
Improvements  can  be  performed  easily  in  real-time  and 
the  effects  can  be  visualized  immediately.  This  bring 
along  advantages  for  the  amount  of  time,  the  quality  and 
the  cost  of  the  development  cycle.  In  Fig.  1 an  example 
for  a virtual  walk-through  in  the  design  process  of  a 
marine  vessel’s  combat  information  center  (CIC)  is 
shown. 

Another  application  is  teleoperation  and  telepresence . 
This  is  not  limited  to  remote  control  of  unmanned 
systems.  Moreover  the  operator  gets  the  subjective 
feeling  of  really  being  there.  This  may  enable  higher 
situational  awareness  which  is  considered  to  be 
advantageous  for  this  application.  Furthermore, 
teleoperation  is  supposed  to  be  useful  for  control  of 
robots  in  contaminated  areas,  space  or  deep  sea  and 
special  military  purposes  (reconnaissance,  surveillance). 


Finally,  the  area  of  education  and  training  is  a field  of 
application.  In  contrast  to  conventional  virtual 
simulation  and  simulators  VE-systems  are  more  flexible 
and  adaptive.  This  is  because  one  single  VE-system  can 
be  used  to  simulate  different  types  of  vehicles  or 
aircrafts.  Moreover,  training  of  individual  soldiers’  skills 
as  well  as  team  training  become  possible. 


Fig.  1:  Virtual  combat  information  center  (CIC) 


The  benefits  of  VE  which  have  been  shown  in  current 
approaches  in  these  areas,  make  it  a promising  tool  for 
further  applications.  One  of  these  applications  is  a 
tactical  situation  display  (TSD)  of  a Command  & 
Control  system. 

3 Tactical  Situation  Displays  (TSD)  today 

The  basic  function  of  TSDs  is  to  display  the  current 
situation  of  own  and  reconnoitered  enemy  troops  and 
facilities  in  the  operation  area  to  the  commander  of  a 
military  unit.  Moreover  the  TSD  is  used  for  tactical 
planning  of  intended  future  operations.  Quantity  and 
quality  of  situation  data  are  essential  for  an  adequate 
operation  planning  (Grand!  et  al,  1997). 

Today  there  are  basically  two  different  types  of  TSD’s 
used  by  the  strike  forces. 

The  first  one,  shown  in  Fig.  2,  is  a command  post  in  the 
field.  The  TSD  used  here  works  by  means  of  paper  & 
pencil.  Actual  information  is  transmitted  by  radio  or 
field  telephone  and  drawn  into  a map.  It  is  obvious  that 
in  time-critical  processes  with  large  amounts  of  rapid 
changing  information  this  leads  to  an  overload  of  the 
operators.  Furthermore,  the  display  may  not  show  actual 
or  valid  information  and  causes  errors  in  decision 
making.  However,  it  brings  along  the  advantage  that  the 
commander  is  in  the  field:  He  gains  high  situational 
awareness,  experiences  the  terrain,  cover,  weather,  etc. 
and  knows  “what  is  really  going  on”  at  that  place. 


Fig.  2:  TSD  at  command  posts  “in  the  field” 
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On  the  other  hand  there  are  TSDs  at  operation  centers , 
as  can  be  seen  in  Fig.  3.  Tactical  situation  data  is 
preprocessed  and  computers  are  used  to  visualize  the 
results. 

The  advantages  of  these  advanced  TSD’s  are: 

• actuality  of  data,  provided  that  the  communication 
infrastructure  is  fast  enough, 

• different  views  of  levels  of  data  aggregation  and 

• possibilities  to  include  additional  battlespace 
information. 

But  this  flood  of  information  may  lead  to  an  information 
overload;  moreover  data  representation  is  still  limited  to 
two  dimensions  and  techniques  of  interaction  with  data 
have  to  be  learnt. 


Fig.  3:  TSD  at  operation  centers 


The  approach  of  using  VE  as  TSD  first  expands  the  two- 
dimensional  visualization  to  three  dimensions.  This 
means  that  height  information  can  easily  be  perceived. 
Additional  elevation  aids,  like  elevation  profiles  or  color 
texturing,  can  be  skipped  and  replaced  by  others  (e.g. 
reconnaissance  photos,  weather  data,  etc. 

The  more  important  thing  is  that  general  interaction  with 
data  is  simplified  and  happens  more  intuitively.  This 
facilitates  an  experience  of  the  tactical  situation  and  the 
generation  of  a correct  mental  model.  In  an  ideal  VE- 
system  the  computer  is  not  realized  as  an  active  entity, 
but  becomes  an  invisible  assistant  which  knows  about 
user  intentions  and  supports  him  (Alexander  et  al., 
1997).  Therefore  operator  workload  is  supposed  to  be 
reduced  and  situational  awareness  to  be  increased. 

4 Approaches  of  VE-Technology  in  C2 

The  amount  of  studies  and  applications  in  the  area  of  VE 
and  VE-technology  has  increased  rapidly  recently.  But 
whereas  VE  is  close  to  become  applicable  in  research 
and  development  and  for  single  training  applications, 
studies  considering  the  specific  use  of  VE  in  Cz  have  just 
begun.  Therefore  knowledge  in  this  area  is  limited  and  a 
lot  of  projects  are  in  a conceptual  phase. 

Most  research  studies  and  projects  in  this  area  have  been 
started  in  the  past  two  years.  Because  of  ongoing 
development  in  this  area  this  is  only  a brief  overview. 
Detailed  information  is  given  in  Alexander  et  al.  (1999). 
Generally  speaking,  the  approaches  can  be  divided  into 
two  groups.  The  first  group  consists  of  concepts  and 
long-term  programs  including  VE-components.  This  is  a 
top-down  approach  which  takes  place  at  high  political 


level  and  typically  application-oriented.  The  second 
group  is  characterized  by  specific  VE-projects  and 
laboratories.  Consequently  it  follows  a bottom-up 
approach  and  is  presentation-  and  technology-oriented. 
Fortunately,  there  are  links  between  both  so  that  they 
meet  and  synergetic  effects  exist. 

The  Swedish  ROLF  (Mobile  Joint  Command  and 
Control  System  2010)  is  a long-term  program.  Its  goal  is 
to  determine  new  possibilities  for  military  commanders 
of  using  VE-Technology  in  mobile  command  posts. 
ROLF  describes  requirements  for  situational  awareness, 
decision  making  and  support,  work  methodology  and 
organization  of  military  crew  and  staff.  The  main  idea  is 
to  use  modem  methods  and  technology  to  help  a group 
of  operators  in  difficult  situations  with  complex,  time- 
critical  decision  making.  ROLF  includes  the  Aquarium 
as  TSD  which  is  a semi-immersive  VE-system.  The  TSD 
is  used  to  visualize  positions  of  own  and  enemy  troops, 
positions  of  important  institutions,  terrain  and  weather 
data  in  different  views.  Data  preprocessing  is  used  to 
select  the  data  displayed  and  ensures  that  only  important 
information  is  visible  (Sundin,  1996). 

Especially  the  realization  that  in  future  battle  scenarios 
all  actions  of  the  military  commander  will  be  in  an 
unclear,  vague  environment  and  the  importance  of  an 
information  dominance  led  to  the  development  of  the 
Command  Post  of  the  Future  Program  (CPoF)  of 
DARPA  (1998).  The  program’s  goal  is  to  accelerate  the 
decision  making  process  with  ongoing  reduction  of  the 
staff.  Therefore  new  technology  is  needed  to  make 
maximum  use  of  the  whole  human  perceptory  system  in 
order  to  transmit  maximum  amount  of  information.  This 
includes  an  interactive,  three-dimensional  visualization, 
three-dimensional  interaction  with  computer-generated 
objects,  presentation  of  inaccuracy  and  probability, 
integration  of  dynamic  factors,  three-dimensional 
symbolic,  integration  of  natural  language  processing  and 
integration  of  knowledge-based  systems. 

The  second,  more  technology-oriented  group  of 
approaches  is  larger.  Institutions  and  laboratories 
working  in  this  area  use  different  VE-technology.  The 
technology  is  often  reconfigured  to  be  used  for  different 
research  projects  and  experiments. 

The  US  Battle  Command  Battle  Lab  (BCBL)  performs 
conceptional  studies  as  well  as  experimental  analyses  in 
a VR-laboratory.  One  goal  is  to  develop  a technology  for 
a multi-media,  scene-based  application  in  education  and 
training  for  organization  and  staff  functions.  This  system 
shall  be  connected  to  the  internet  to  increase  the  range  of 
application  (Heredia,  1999). 

At  the  US  Naval  Research  Laboratory  (NRL)  an 
advanced  battle  planing  and  management  system  has 
been  developed.  The  system  works  with  a semi- 
immersive  display  and  enables  multi-modal  interaction. 
It  was  found  to  be  very  suitable  for  virtual-prototyping, 
interactive  mission  planing  and  increasing  situational 
awareness  (NRL,  1997). 

Similar  approaches,  like  Mirage  of  the  Army  Research 
Lab  (ARL)  (1ST,  1997),  the  Visualization  Architecture 
Technology  (VAT)  of  the  Crewstation  Technology 
Laboratory  (CTS)  (Achille,  1998)  or  the  Electronic  Sand 
Table  of  MITRE  Corp.  (MITRE,  1998)  also  use  a semi- 
immersive  VE-technology,  as  described  further  on. 
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Other  approaches  use  full-immersive  VE  or  desktop- VE 
respectively  (Dockery  & Hill,  1996;  Morgenthaler  et  al. 
1998). 

5 The  Electronic  Sandtable  (ELSA) 

The  Electronic  Sandtable  has  been  developed  as  an 
advanced  display  for  tactical  information  in  mission 
planing,  control  and  rehearsal.  The  concept  is  based  on 
the  sandtable  metaphor.  The  military  sandtable,  as 
shown  in  Fig.  4,  consists  of  a sandy  model  landscape 
with  simplified  objects  representing  woods,  buildings, 
points  of  interest  or  military  units.  It  is  broadly  used  in 
military  education  and  training. 


Fig,  4:  Sandtable  in  military  education 


But  the  traditional  sandtable  is  static;  all  changes  of 
deployment  have  to  be  done  manually.  Each  change  of 
region  is  very  time-consuming  and  has  also  to  be  done 
manually.  Moreover  the  accuracy  for  representing  real 
geographic  data  is  poor. 

It  is  intended  to  model  the  sandtable  by  means  of  a VE- 
system.  This  way  dynamics,  real-time  interaction  and 
changes  of  the  point-of-view  can  be  included  while  the 


benefits  of  the  real  sandtable  remain. 

For  this  purpose  geographic  data  and  tactic  data  have  to 
be  visualized  stereoscopically.  It  is  intended  to  create  a 
model  landscape,  in  which  dynamic  battle  scenario  is 
included.  Furthermore  additional  functionality  is  added, 
e.g.  visibility,  range  of  weapon  systems,  etc. 

5.1  Structure  and  Technical  Implementation 

The  Electronic  Sandtable  has  been  implemented  as  a 
testbed  at  the  Research  Institute  for  Communication, 
Information  Processing  and  Ergonomics  (FKIE).  A draft 
of  the  technical  setup  is  shown  in  Fig.  4. 

Because  of  the  large  size  of  geographic  databanks  and 
the  need  for  real-time  interaction,  the  underlying 
structure  has  been  arranged  in  two  stages  (Alexander  et 
al,  1997). 

The  first  stage  is  executed  offline.  In  this  stage  the  scene 
graph  is  determined.  The  scene  graph  is  a hierarchically 
ordered  databank  of  all  polygons  included  in  the  visible 
scene. 

Originally  all  data  is  separated  in  different  databanks, 
which  means: 

• digital  terrain  elevation  data  (DTED), 

• digital  feature  analyses  data  (DFAD), 

• textures  (e.g.  reconnaissance  pictures), 

• single  geometric  objects  (buildings,  tanks,  airplanes) 
and  their  attributes. 

In  a semi-automatic  process  data  and  objects  are 
selected,  integrated  and  re-ordered  with  respect  to 
maximum  rendering  performance.  This  databank  is 
called  the  scene  graph.  Afterwards  the  structure  of  the 
scene  graph  stays  constant  without  any  changes. 

In  the  second  stage  additional  data  is  constantly  added 
and  the  scene  graph  is  visualized  online.  The  additional 
data,  i.e.  tactical  situation  data  and  data  from  external 
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data  sources,  is  linked  to  objects  of  the  scene  graph. 
Additional  input  of  external  data  using  different 
protocols  (DIS,  HLA)  shall  also  become  possible  in 
future.  The  incoming  data  controls  position  and  status  of 
military  units.  Additional  data,  like  actual  situation 
videos  or  information  of  knowledge  databanks,  can  also 
be  included. 

After  that  the  rendering  subsystem  selects  the  visible 
subset  of  the  scene  graph.  Out  of  this  two  separate 
projections  are  calculated  and  written  into  the  two  frame 
buffers.  Then  both  frame  buffers  are  visualized 
alternately  on  a horizontal  plane. 

The  human  operator  interacts  with  the  scene  by  means  of 
different  interaction  devices.  The  inputs  serve  as 
commands  which  affect  the  objects  of  the  scene  graph. 
The  actions  are  logged  for  later  analysis. 

The  operator  is  able  to  select  different  visible  areas  for 
navigation.  The  borders  of  the  area  serve  as  one  input 
variable  of  the  rendering  subsystem.  Additionally  each 
of  the  operator’s  movements  is  tracked  by  a head- 
tracker.  The  position  output  of  the  tracker  is  another 
input  variable  of  the  rendering  subsystem  for  new 
projection  calculation. 

5.2  Generation  of  the  Scene  Graph 

The  scene  graph  is  the  output  of  the  first  stage  described 
in  chapter  5.1.  The  generation  process  itself  is  offline 
and  semi-automatic.  As  far  as  possible  COTS-products 
are  used. 

The  process  is  divided  into  a data  selection, 
preprocessing  and  optimizing  phase. 

Preprocessing  and  optimizing  are  necessary  because 
terrain  and  feature  data  are  generated  from  geographic 
databanks.  These  databanks  were  designed  with  regard 
to  different  requirements  which  makes  them  unsuitable 
for  a real-time,  realistic  visualization. 

5.2.1  Data  Selection 

In  the  first  step  an  area  of  interest  is  selected  and  the 
relating  terrain  (DTED)  and  feature  (DFAD)  data  is 
extracted.  Additionally,  links  between  features  and 
geometric  objects  are  defined.  Afterwards  the  selected 
data  is  saved  in  a temporary  buffer  which  has  to  be 
preprocessed  and  optimized  for  visualization. 

The  geographic  data  available  is  divided  into  (Helmuth, 
1996): 

• Raster  data : pixel  data  (e.g.  scanned  paper  maps), 

• Picture  data:  geo-referenced  or  non-referenced  aerial 
or  satellite  photos. 

• Vector  data:  surfaces  (e.g.  woods,  lakes),  lines  (e.g. 
streets,  rivers)  or  points  (e.g.  power  poles,  points  of 
interests,  bridges,  towers)  with  the  position  of  their 
bases  and  attributes.  For  visualization  vector  data  is 
linked  to  detail  objects. 

• Matrix  data:  data  in  matrix  format  of  a specific 
resolution.  Usually,  terrain  data  is  organized  like  this. 

Geometric  objects  are  components  which  are  linked  to 
geographic  data,  especially  geographic  vector  data,  and 
tactical  situation  data.  They  include  a geometric 
description  of  the  object  (e.g.  tanks,  airplane)  and 
additional  information  (e.g.  unit  status,  damage  reports, 
etc.).  At  the  stage  of  real-time  visualization  they  are 


shown  at  the  position  given  either  by  the  geographic 
data  or  the  tactical  situation  data.  General  geometric 
objects  are  often  automatically  constructed.  However, 
more  sophisticated  models  have  to  be  designed  manually 
either  by  a CAD-program,  a modeling  software  or  out 
of  an  object  databank. 

For  later  selection  operations  attributes  have  to  be 
added.  Attributes  can  be  divided  into  geometric  and 
general  attributes.  While  geometric  attributes  (length, 
width,  height)  are  the  same  for  each  object,  general 
attributes  are  dependent  on  the  kind  of  application.  Such 
attributes  might  be  population  data  (sociology),  pollutant 
emission  (environment)  or  tactical  information 
(military). 

5.2.2  Data  Preprocessing 

Consistency  and  integrity  are  highly  important  criteria 
for  databanks.  If  datasets  of  more  than  one  databank  are 
merged,  contradicting  data  might  emerge  and  cause 
errors.  Those  errors  are  based  on  errors  or  inaccuracies 
in  the  original  databanks,  different  data  resolution  or 
different  actuality  of  data  acquisition. 

As  soon  as  consistency  and  integrity  is  proved,  the 
process  of  merging  terrain  and  feature  data  starts. 
Geometric  objects  are  appended  and,  if  necessary, 
adjusted  to  ground  level. 

Finally  the  triangulation  process  starts  and  determines 
polygons  for  visualization. 

5.2.3  Data  Optimizing 

For  real-time  visualization  the  amount  of  rendered 
polygons  has  to  be  minimal.  Therefore  the  databank 
system  transfers  only  information  about  the  visual 
subset.  Non-visible  parts  outside  the  field  of  view  are 
clipped. 

For  further  reduction  the  databank  is  re-organized  and 
the  scene  graph  is  tiled.  In  the  visualization  process  the 
distance  to  the  point  of  view  sets  the  level  of  complexity 
for  each  tile. 

Different  levels  of  complexity,  also  called  levels  of 
detail  (LOD),  are  another  technique  to  reduce  polygons. 
LOD  means  more  than  one  representation  of  different 
levels  of  complexity  (different  amount  of  polygons)  for 
the  same  subset.  This  means,  if  a subset  gets  closer  to 
the  point  of  view,  a higher  LOD  with  more  polygons  is 
visualized. 

Using  these  techniques  data  is  re-organized  with  regard 
to  visualization  issues.  The  output  of  this  process  is  the 
scene  graph  which  is  visualized  in  real-time  in  the 
second  stage. 

5.3  Display  Technology 

The  display  technology  used  for  three-dimensional 
visualization  is  a semi-immersive  virtual  workbench. 
This  concept  has  originally  been  developed  by  Kruger  & 
Frohlich  (1992).  The  baseline  concept  is  shown  in  Fig. 
5.  Today  it  is  used  for  various  applications. 

A projector  projects  two  computer-generated,  time- 
alternated  pictures  onto  a mirror.  The  mirror  reflects 
them  to  a horizontal  focussing  screen.  By  using  shutter 
glasses,  i.e.  LCD-glasses  shading  each  side  alternately 
synchronous  to  the  projection,  the  operators  perceive 
two  separate  pictures  for  the  right  and  the  left  eye.  The 
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synchronization  works  by  an  emitter  sending  infrared 
signals  synchronously  to  the  picture  projected. 


Operators  wearing 
Shutterglasses 


Projector 


Virtual  Workbench 


Fig.  5:  Principle  of  a semi-immersive  virtual  workbench 


Finally,  both  pictures  perceived  are  fusioned  by  the 
cerebrum  to  a single,  three-dimensional  model. 


6 User  Interface 


• Subsystem  2,  jointly  with  the  central  nervous  system, 
transfers  the  light  to  stimulus  signals  of  nerve  cells.  It 
consists  of  the  retina  with  its  two  different  light 
receptors. 

The  stimuli  are  transferred  via  the  optic  nerve  to  the 
optic  centers  of  the  cerebrum.  Here  the  optic  sensing  and 
recognition  takes  place. 

Visual  perception  is  generally  based  on  three  stages  of 
perception  (Kelle,  1994): 

The  first  stage  is  an  egocentric  perception  of  the  own 
person.  This  allows  a separation  of  objects  of  the  own 
body  and  other  objects,  making  possible  to  determine  the 
own  position  with  regard  to  other  objects  and  an 
absolute  depth  perception. 

The  next  step  is  a comparision  of  the  objects  in  the 
environment,  allowing  a relative  depth  perception. 
Finally  memory,  experience  and  internal  processing 
mechanism  lead  to  depth  cues  being  fundamental  for 
spatial  perception. 


The  design  of  the  user  interface  of  VE-systems  has  been 
found  to  be  one  of  the  main  criteria  of  quality  for  its 
application.  The  Electronic  Sandtable  serves  as  the 
interface  between  the  real  environment  on  the  one  hand 
and  the  virtual  scene  on  the  other  hand.  Moreover  it  uses 
a different  metaphor  than  the  desktop-metaphor  used  in 
various  computer  applications.  Therefore  new 
interaction  techniques  and  procedures  have  to  be 
developed,  analyzed  and  optimized  according  to  a high 
performance  of  the  human-computer-system  (Alexander, 
1999). 

This  includes  visualization,  interaction  and  cooperation 
in  a virtual  scene. 

6.1  Visualization 

A realistic,  three-dimensional  visualization  of  terrain 
data  has  to  consider  the  physiological  procedures  of 
visual  depth  perceiving.  These  procedures  have  been 
studied  extensively,  and  several  different  hypothesis  for 
depth  perceiving  exist. 

Each  hypothesis  postulates  the  existence  of  depth  cues. 
The  classic  depth  cues  will  be  summarized  later  in  this 
chapter.  Especially  the  depth  cues  of  stereoscopic 
disparity  and  parallax  are  of  critical  importance  for  the 
application  of  the  Electronic  Sandtable. 

A computer-based  visualization  has  to  take  into  account 
different  depth  cues.  For  stereoscopic  visualization 
different  viewing  models  exist.  The  common  models 
will  be  presented  in  this  chapter  as  well. 

6.1.1  Visual  Perception 

The  physiological  visual  system  consists  of  the  eye  as 
sense  organ  for  stimulus  acquisition,  the  optic  nerve  for 
stimulus  transfer  and  the  optic  center  of  the  cerebrum  for 
stimulus  processing. 

According  to  Schmidt  & Thews  (1995)  the  human  eye 
can  be  divided  into  two  subsystems: 

• Subsystem  1 is  responsible  for  the  refraction  of 
incoming  light.  Its  main  components  are:  Iris  (control 
of  incoming  light  intensity),  lens  (refraction), 
vitreous  body  (stability)  and  diverse  muscles 
(adjustment). 


6.1.2  Depth  Cues 

Depth  cues  are  visual  system  cues  which  enable 
perceiving  of  spatial  dependencies  (Hodges,  1992; 
Schmidt  & Thews,  1995).  The  can  be  divided  into 
monocular  and  binocular  cues. 

Monocular  cues  are  valid  for  perception  with  one  eye 
only. 

The  main  monocular  cues  are: 

• perspective, 

• difference  in  size, 

• known  dimensions  of  objects, 

• shading, 

• light  and  shadow, 

• accommodation. 

The  binocular  depth  cues  require  the  total  binocular  eye 
system.  They  influence  the  perception  of  short  to 
medium  distances. 

Traditional  binocular  depth  cues  are: 

• convergence:, 

• disparity  and  parallax. 

Additionally  to  these  static  cues  there  are  dynamic  cues 
which  have  large  influences  on  the  depth  perception  for 
medium  distances  (17  -29  m)  (Kelle,  1994). 

6.1.3  Disparity  and  Parallax 

Disparity  and  parallax  have  a large  influence  on  depth 
perception  and  are  the  main  depth  cues  for  stereoscopic 
visualization.  Therefore  they  are  described  more 
detailed. 

The  distance  between  both  eyes  leads  to  different 
representations  of  an  object  on  the  retina  of  the  right  and 
the  left  eye.  Both  eyes  perceive  the  object  with  a 
different  perspective.  The  difference  between  both 
pictures  is  described  by  the  disparity. 

If  an  object  is  focussed,  it  is  represented  at  the  fovea  of 
both  eyes.  A round  spatial  surface  exists  (horopter), 
representing  all  objects  on  it  on  corresponding  retina 
areals.  Objects  at  positions  different  from  the  horopter 
are  represented  at  non-corresponding  retina  areas.  If  the 


12-7 


distance  from  the  horopter  is  not  too  large,  the  cerebrum 
fusions  the  right  and  left  picture  to  a three  dimensional 
model.  If  it  is  too  large,  disturbing  double  pictures  are 
perceived  (Schmidt  & Thews,  1995). 

Disparity  is  a mathematical  dimension  and  cannot  be 
determined  practically.  Therefore  the  dimension  of  the 
stereoscopic  parallax  has  been  introduced.  For  this  a 
reference  level  has  been  used  which  is  parallel  to  the 
eyes’  level  and  runs  through  the  fixation  point. 

Parallax  has  been  defined  as  (Helmholtz,  1910,  ref.  in: 
Kelle,  1994): 


P = ba 


xax 


t 

e * t + e2 


p = parallax 

b.,  = inter  ocular  distance 

a = distance  eyes  / reference  level 

e = distance  reference  level  / object 

t = distance  eyes  / object  ( =a+e  ) 

Parallax  is  also  a dimension  for  depth  separation  and 
depth  perception.  Therefore  it  is  deduced  that  depth 
perception  decreases  with  square  distance.  Furthermore 
it  increases  linearly  with  inter  ocular  distance. 

According  to  Kelle  (1994),  stereoscopic  disparity  and 
parallax  has  been  found  to  be  useful  only  for  near  and 
medium  distance  (maximum  of  6-9  m). 

Visualization  of  geographic  data  of  large  scale  means  a 
large  distance  between  eye  point  and  surface.  It  can  be 
concluded  that  exact  modeling  means  that  parallax  and 
stereoscopic  depth  perception  will  be  very  low.  Instead 
inter  ocular  distance  has  to  be  modified  to  several  meters 
evoking  a higher  stereoscopic  depth  perception. 


6.1.4  Stereoscopic  Projection  Models 

For  three-dimensional  stereoscopic  visualization  three 
different  projection  models  are  commonly  used.  Their 
baseline  geometry  is  illustrated  in  Fig.  6. 
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Fig.  6:  3 projection  models:  (a)  parallel  projection, 
(b)  rotated  projection,  (c)  window  projection 


In  Computer  Aided  Design  (CAD),  aerial  photo  analysis 
and  for  head-mounted-displays  (HMD)  projection 
models  with  parallel  line  of  sights  are  used,  as  shown  in 
Fig.  6 (a).  They  are  based  on  the  assumption  of  a center 
eye-point  perpendicular  to  the  projection  plane.  Right 
and  left  projection  are  calculated  by  using  offset  values 
and  parallel  shifting  the  projection  right  and  left.  The 
disadvantage  of  this  model  is  that  the  scene  can  only  be 
visualized  underneath  the  projection  plane.  This  is 
inconvenient  for  the  concept  of  the  Electronic  Sandtable, 
because  the  scene  would  always  be  located  beyond  hand 
range.  Another  disadvantage  is  clipping  at  the  borders  of 
the  display  as  there  are  missing  visual  information  for 


either  the  right  or  the  left  eye.  Especially  at  large 
displays  this  is  very  irritating  for  operators. 

Fig.  6 (b)  shows  the  geometry  of  a projection  model 
using  rotated  line-of-sights.  Here  the  projections  are 
rotated  in  the  way  that  both  lines-of-sight  meet  in  the 
projection  plane.  The  lines-of-sight  are  not 

perpendicular  to  the  projection  plane.  The  concept 
remains  the  same,  no  matter  if  the  scene,  the  eye  points 
or  both  are  rotated.  It  enables  a visualization  underneath 
and  as  well  as  above  the  projection  plane.  There  are  no 
irritating  effects  on  the  borders  of  the  display  either.  But 
because  of  the  special  geometry,  an  error  of  vertical 
parallax  appears.  This  can  be  observed  especially  at  the 
borders  of  the  display,  where  both  lines  meet  at  a point 
above  the  projection  plane.  This  leads  to  a “winding”- 
effect  and  the  scene  seems  to  be  projected  on  a cylinder 
rather  than  a plane.  The  error  is  perceived  especially  on 
large  displays. 

The  last  projection  model  uses  window  projection, 
which  means  that  two  windows  are  introduced  through 
which  the  virtual  scene  is  perceived.  The  windows  are 
positioned  in  the  same  level  as  the  projection  plane. 
Both  lines-of-sight  meet  at  the  projection  plane  and 
remain  perpendicular  to  it.  This  is  correct  for  the  middle 
and  the  border  of  the  projection  area.  In  this  model, 
stereoscopic  parallax  is  only  dependent  on  the  distance 
to  the  display  and  no  vertical  parallax  is  introduced. 

This  model  is  used  for  the  Electronic  Sandtable.  As 
shown  in  Fig.  7,  the  model  for  each  eye  is  described  by 
an  asymmetric  pyramid.  This  means,  the  perpendicular 
line  through  the  top  docs  not  meet  the  center  of  the 
pyramid  basis. 


Fig.  7:  Right  and  left  asymmetric  projection  pyramid 
and  boundary  surfaces  (clipping  planes) 


For  each  projection  six  parameters  are  used  to  identify 
the  pyramid.  They  include  the  values  for  front,  back,  top, 
bottom,  left  and  right  clipping  plane.  These  values  are 
calculated  by  x,y,z-position  of  both  eye-points,  scale 
factor  and  the  display  size  as  input. 

Pilot  experiments  have  shown  good  results  for  this 
projection  model.  Only  little  perspective  error  due  to 
tracking  of  the  real  eye  position  was  determined.  In 
future,  this  error  will  be  minimized  by  calibrating  the 
tracking  equipment. 

6.1.5  Future  Research  in  Visualization 

So  far  only  real-size  shapes  have  been  visualized.  In 
future  geographic  data  of  different  scales  will  be  used. 
To  evoke  a stereoscopic  depth  perception,  an  adaption  of 
the  scale  factor  for  elevation  as  well  as  the  dimension  of 
inter  ocular  distance  is  necessary. 
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However,  the  adaption  may  lead  to  either  too  flat  or  too 
heigth  depth  perception.  Both  is  not  wanted  and 
therefore  research  studies  will  be  done  concerning  the 
correct  determination  of  both  parameter. 

Another  research  topic  is  the  maximum  vertical  range  of 
the  display.  The  display  technique  causes  contradicting 
depth  information,  because  both  eyes  accomodate  on  the 
projection  plane,  but  fixate  an  object  closer  or  more  far 
away.  However,  if  the  virtual  scene  is  too  close,  parallax 
becomes  too  large  and  the  cerebrum  cannot  fusion  both 
pictures.  Therefore  another  research  topic  will  be  to 
determine  the  maximum  useful  vertical  display  range 
and  the  variability  of  human  sense  perceiving. 

Pilot  experiments  in  this  area  have  been  started  and  are 
currently  going  on. 

6.2  Interaction 

Interaction  with  the  databank  means  navigation  in  the 
scene  and  manipulation  of  virtual  objects.  For  both 
subgroups  procedures  (software)  and  interaction  devices 
(hardware)  have  to  be  designed,  evaluated  and  analyzed 
according  to  the  application. 

6.2.1  Navigation 

Navigation  can  be  divided  into:  Navigation  within  the 
databank  system  and  navigation  within  space. 

The  first,  navigation  within  the  databank  system, 
encloses  different  procedures  for  search  and  selection  of 
datasets.  This  is  a general  topic  and  is  not  of  special 
interest  for  VE  systems.  Therefore  it  will  not  be 
described  in  this  paper. 

Navigation  within  space  means  a change  of  position  or 
orientation  of  the  observer.  This  is,  generally  speaking,  a 
modification  of  display  area.  A first  implementation  sets 
a starting  position  and  orientation  for  the  observer.  Both 
can  be  modified  by  user  input.  There  are  many 
possibilities  for  designing  the  navigation  procedure  of 
the  graphic  user  interface  (GUI). 

One  is  an  adaption  of  the  procedures  used  by  MS 
Windows®  or  OS  Motif®.  An  example  of  this  can  be 
seen  in  Fig.  8.  In  this  concept,  the  operator  has  to  turn 
dials  or  move  sliderbars  at  the  sides  of  a 
software-window  to  modify  the  display  area.  For 
desktop  computers  this  concept  is  familiar,  often  used 
and  nearly  standard.  However,  on  large  displays  it  was 
found  to  be  very  inconvenient.  The  reason  for  this  is  that 
reaching  for  the  navigation  control  means  to  move  a 
cursor  a long  way  across  the  display  and  this  takes  a lot 
of  time. 


Fig.  8:  Different  desktop  navigation  concepts, 
left  side:  MS  Windows  ® and  sliderbars; 
right  side:  Cosmoplayer®  and  virtual  cockpit 

Secondly,  concepts  for  “virtual  cockpits”  are  possible,  as 
shown  in  Fig.  8.  The  operator  moves/flies  through  the 


scene  by  controlling  a throttle  and  a virtual  control  stick 
shown  in  the  lower  middle  of  the  GUI.  This  technique  is 
often  used  for  desktop-VE  systems,  especially  for 
VRML-applications.  However,  for  larger  displays  and 
semi-immersive  VE-systems  it  has  been  found  to  be  not 
efficient  enough. 

For  reasonable  use  of  the  benefits  of  VE,  the  interaction 
has  to  be  more  intuitively.  The  concept  of  “grab-and- 
move”  describes  a first  step  in  this  direction.  In  this 
concept  the  operator  grabs  the  scene  and  slides  it  to  any 
direction.  First  trials  using  an  implementation  of  this 
concept  show  that  navigation  becomes  rather  easy  and 
fast. 

In  the  actual  experimental  setup  a trackball  is  used  as 
main  interaction  device.  Missing  degrees  of  freedom  are 
simulated  by  additional  switches  on  the  device.  For 
future  studies  evaluations  of  other  devices  are  planned. 
One  of  those  will  be  a virtual  laser  pointer  which  makes 
pointing,  selection  and  navigation  possible. 

6.2.2  Manipulation 

Manipulating  the  objects  of  the  virtual  environment  will 
use  the  same  devices  as  for  navigation.  A switch  or 
control  bar  is  used  to  toggle  between  navigation  and 
manipulation  mode.  In  manipulation  mode  the  operator 
has  to  be  able  to  select  objects  and  choose  special 
control  operations. 

Main  control  operations  are: 

• generation  of  new  objects, 

• editing  attribute  values, 

• erasing, 

• placement, 

• orientation, 

• movement, 

• editing  terrain  data. 

Generation,  editing  and  erasing  of  objects  can  easily  be 
implemented.  But  placement,  orientation,  movement  and 
editing  of  terrain  data  are  more  difficult,  because 
interactions  between  different  objects  of  the  scene  graph 
have  to  be  considered.  This  can  easily  lead  to  errors  in 
the  structure  of  the  scene  graph. 

For  correct  placement  and  orientation  of  objects  in  the 
three-dimensional  space  elevation  information  about  the 
base  point  is  necessary.  Moreover,  if  the  object  is  placed 
on  oblique  terrain,  it  has  to  be  oriented  correctly.  This 
information  can  only  be  derived  from  the  original 
geographic  databank  which  is  not  a part  of  the  scene 
graph.  For  this  reason  data  exchange  with  the  original 
geographic  databank  has  to  be  possible. 

Movement  of  objects  is  even  more  critical.  It  describes 
the  process  of  detaching  the  object  out  of  the  scene 
graph  and  re-placing  it  at  another  position.  A trivial 
implementation  would  be  to  erase  the  old  object  and 
generate  an  identical  new  object  at  the  new  place.  But 
intelligent  movement  algorithms  should  enable  a 
movement  of  the  object  while  simultaneously  fixating  it 
on  the  ground  level  {terrain  following). 

More  complex  edit  operations  of  the  terrain  databank  are 
very  difficult,  because  the  original  geographic  databank 
has  to  be  modified,  stored  and  re-converted  into  a new 
scene  graph.  As  said  before,  this  has  to  happen  offline. 
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Consequently  the  actual  setup  of  the  Electronic 
Sandtable  does  not  support  these  operations. 

6.3  Cooperation 

The  concept  of  the  Electronic  Sandtable  has  been 
designed  to  enable  multiple  operator  working  in  the 
virtual  scene.  It  has  to  include  cooperation  concepts. 

In  contrast  to  full-immersive  VE,  in  semi-immersive  VE 
all  operators  are  present  at  the  same  location. 
Communication  and  inter-operator  interaction  happens 
the  natural  way.  Therefore  mainly  human-computer 
interaction  issues  have  to  be  analyzed.  These  main  issues 
and  problems  will  be  discussed  in  the  following. 

A correct  perspective  visualization  of  the  computer- 
generated picture  is  limited  to  only  one  single  operator 
due  to  technical  reasons.  The  technically  possible  frame 
rate  is  limited  to  a maximum  of  144  Hz.  For  stereoscopic 
visualization  the  rate  is  cut  into  halves  (72  Hz).  An 
additional  operator  would  mean  to  cut  it  into  halves 
again  (36  Hz),  making  flickering  as  well  as  occurring  of 
hazardous  separation  of  single  pictures  likely  to  appear. 
In  a word  it  can  be  summarized  that  with  today’s 
technology  only  calculation  and  stereoscopic 
presentation  for  a single  operator  is  possible.  Further 
operators  will  perceive  a perspective  error. 

One  way  of  dealing  with  the  problem  would  be  to  track 
more  than  one  operator  and  to  calculate  an  average 
position.  Average  position  does  not  necessary  mean  the 
arithmetic  mean,  it  could  be  a more  complex  formula 
determined  by  subjective  ratings.  This  way  the 
perspective  error  might  be  minimized  and  might  not  be 
subjectively  perceivable. 

Apart  from  visualization,  cooperation  also  has  effects  on 
interaction  and  manipulation.  For  different  operators 
working  in  a virtual  scene,  two  main  concepts  exist.  The 
concepts  are  drafted  in  Fig.  9. 

The  conference  concept  is  characterized  by  an  active 
presenter  and  passive  participants.  The  presenter  is  able 
to  navigate  and  manipulate,  whereas  the  participants  are 
following  the  presentation.  There  is  no  real  cooperation 
taking  place. 

On  the  other  hand,  in  the  workshop  concept , all 
participants  become  active.  However,  they  are  given 
different  rights  for  access.  At  the  beginning  of  each 
session,  an  operational  area  is  set  as  working  region. 
Afterwards  participants  are  able  to  work  in  this  region. 
This  concept  is  not  limited  to  human-computer 
interaction  but  has  to  include  cooperation  procedures 
between  session  participants  as  well. 


Fig.  9:  Conference  (left)  and  workshop  (right)  concept 

A typical  example  is  handing  over  an  object.  Participant 
#1  selects  an  object  and  gets  access  rights  for  it.  For 
exact  placement  he  hands  it  over  to  participant  #2. 


Access  rights  move  over  to  participant  #2  as  well. 
Finally  participant  #2  drops  the  object  and  loses  access. 

These  concepts  require  the  introduction  of  new 
procedures  and  an  intense  research  in  this  area.  Because 
reality  is  still  to  be  modeled  natural  procedures  serve  as 
input  for  the  model.  But  just  modeling  reality  does  not 
consider  that  VE-systems  have  much  more  capabilities. 
Therefore  new  advanced  concepts  have  to  be 
formulated,  to  optimize  use  and  gain  benefits  of  the 
system. 

7 Conclusion  and  Future  Research 

In  this  paper  the  baseline  concept  of  using  semi- 
immersive  VE-technology  as  advanced  TSD  has  been 
described.  The  approach  has  been  shown  to  be 
promising  and  advantageous. 

It  has  been  emphasized  that  human  factors  and 
ergonomics  are  the  main  issues  for  reasonable 
VE-application.  Main  research  issues  were  found  to  be 
visualization,  interaction  and  cooperation.  But  these 
topics  cannot  be  analyzed  separately,  because 
interactions  between  them  exist.  In  this  paper  some 
research  issues  were  introduced  and  results  of  ongoing 
research  studies  in  the  area  of  visualization  were 
presented. 

But  even  if  in  future  the  system  works  as  it  is  supposed 
to  be,  one  question  to  be  answered  still  remains:  The 
question  for  quantification  of  the  profit  and  gain  of  using 
VE-systems.  The  key  criteria  for  answering  this  question 
will  be  performance  of  the  human-VE  system. 

For  this  reason  human  performance  metrics  will  have  to 
be  introduced,  formulated  and  analyzed.  They  should  be 
as  fundamental  as  possible,  but  still  take  into  account  the 
characteristics  of  the  application. 

Jointly  with  other  basic  research  studies  they  will  be  the 
key  issues  of  future  research  in  this  area. 
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